Skip to content

Update embed-code to v1.2.4 so checkSamples no longer passes silently - #84

Merged
alexander-yevsyukov merged 12 commits into
masterfrom
claude/nifty-northcutt-1537c8
Oct 2, 2026
Merged

alexander-yevsyukov merged 12 commits into
masterfrom
claude/nifty-northcutt-1537c8

Conversation

@alexander-yevsyukov

@alexander-yevsyukov alexander-yevsyukov commented Jul 24, 2026 •

Copy link
Copy Markdown
Contributor

Summary

./gradlew checkSamples — and CI's Check Code Embedding workflow — was passing while verifying nothing. Two problems compounded:

  1. Stale config. Update project folder structure #71 moved the main (v1) documentation from content/docs/1 to the root content/docs, but docs/_settings/v1.embed-code.yml kept pointing at ../content/docs/1, which no longer exists.
  2. Swallowed error. The committed embed-code binaries were v1.2.0, which logs argument errors like that missing path but exits with status 0. Upstream fixed this in SpineEventEngine/embed-code-go@9fa4244, first released in v1.2.3: every failure now exits with status 1.

This PR updates the binaries to v1.2.4 (the latest release) and points the config at the root docs, so checkSamples checks the real content again. It also adds the missing config step to the main-version switch in SPINE_RELEASE.md, so the next content move doesn't break the check silently. Finally, it picks up a config update that adds Gradle Wrapper validation.

An earlier revision of this PR worked around the swallowed error with a fail-closed guard in docs/_script/check-samples. With v1.2.4 the guard is redundant, so it was removed and the script is unchanged from master.

Changes

  • docs/_bin — binaries updated from v1.2.0 to v1.2.4, plus the Windows binary for completeness. Each file matches the SHA-256 digest published with the release.

    Release asset File in docs/_bin
    embed-code-linux embed-code-ubuntu (used by CI)
    embed-code-macos-arm64.zip embed-code-macos
    embed-code-macos-x64.zip embed-code-macos-x86_64
    embed-code-windows.exe embed-code-windows.exe (new; no script invokes it)
  • docs/_settings/v1.embed-code.yml — docs-path now points at ../content/docs. The embed-mappings block was dropped: v1.2.3 renamed that key to embeddings, so v1.2.4 ignored it and read only the top-level paths.

  • SPINE_RELEASE.md — the "Change the current main version" procedure gains a step: when the content moves, update docs-path in the embed-code config.

  • docs/assets/js/docs/architecture-diagram.js — a comment now points at the architecture page's current path, /docs/introduction/architecture.md.

  • config — submodule bumped 3210ca6 → 94a9e08 (6 commits), bringing in Keep Gradle Wrapper validation in Hugo-only repositories config#781 (keep Gradle Wrapper validation in Hugo-only repositories) and Point to the site for verifying published artifacts config#780 (point to the site for verifying published artifacts).

  • .github/workflows/gradle-wrapper-validation.yml — new, distributed by that config update. It validates the Gradle Wrapper JAR with gradle/actions/wrapper-validation on pushes to master and on every PR. This repo keeps the wrapper only to run Hugo through Gradle, and none of its workflows uses setup-gradle (which would validate the wrapper itself), so nothing else checked that the JAR is genuine.

Verification

embed-code check mode, run locally with the macOS binaries:

Scenario v1.2.0 (before) v1.2.4 (after)
Config file missing exit 0 ❌ exit 1 ✅
docs-path does not exist exit 0 ❌ exit 1 ✅
Valid config, embeddings up to date exit 0 exit 0
Stale embedding exit 1 (raw panic) exit 1 (clean error)
Missing code files exit 1 (raw panic) exit 1 (clean error)

With the final config and v1.2.4, through the Gradle tasks:

  • ./gradlew :checkSamples passes: "The documentation files are up-to-date with code files."
  • With one embedding deliberately made stale, ./gradlew :checkSamples fails with exit status 1: "Check failed: the documentation files are not up-to-date with code files."
  • ./gradlew :embedCode finishes without changing any file, so no embeddings needed regenerating.

The Linux binary already ran on CI for this PR and failed correctly, with exit status 1, on the old missing path. The Windows binary was not run.

Notes for reviewers

  • Keep the binaries in docs/_bin at v1.2.3 or later: older versions bring the silent pass back.

🤖 Generated with Claude Code

`./gradlew checkSamples` was exiting 0 while verifying nothing, so CI's
"Check Code Embedding" workflow stayed green regardless of embedding drift.
Two compounding bugs:

1. Stale config: `docs/_settings/v1.embed-code.yml` set `docs-path` to
   `../content/docs/1`, a directory removed in the site restructure. The
   embed-code binary rejected it with an argument-validation error.

2. Swallowed failure: on argument-validation errors the embed-code binary
   prints an ERROR line but still exits 0, so `set -e` in
   `docs/_script/check-samples` had no non-zero status to trip on.

Changes:
- Repoint `docs-path` to `../content/docs` (top-level key and embed-mappings).
- Harden `check-samples` to fail closed: the check now passes only when the
  binary exits 0 AND prints its success sentinel ("documentation files are
  up-to-date with code files"). An argument error (exit 0), a stale embedding,
  or any other silent-exit-0 failure is now reported and fails the build.

Verified via `./gradlew :checkSamples` (what CI runs): passes on a clean tree,
and now fails with exit 1 both on a deliberately stale embedding and on a bad
`docs-path`. The embeddings themselves were already up to date, so no
regeneration was needed.

The root cause (exit 0 on argument errors) lives upstream in
SpineEventEngine/embed-code-go; the guard protects this repo regardless.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alexander-yevsyukov alexander-yevsyukov self-assigned this Jul 24, 2026
@alexander-yevsyukov alexander-yevsyukov moved this to 🏗 In progress in v2.0 Jul 24, 2026
Reverts the docs-path change from the prior commit on this branch. Per the
project's documentation versioning, the v1 embed-code config stays pointed at
the `/1` subfolder, so the net change of this branch is just the fail-loud
guard in check-samples.

Because commit b184e4d ("Move the main documentation version to the root")
moved v1 content to the root and no docs/content/docs/1 directory exists yet,
checkSamples now fails loudly via that guard, reporting the missing path.
Restoring `/1` is handled separately.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alexander-yevsyukov alexander-yevsyukov changed the title Fix silently-passing checkSamples code-embedding check Make checkSamples fail loudly on embed-code errors Jul 24, 2026
@alexander-yevsyukov alexander-yevsyukov moved this from 🏗 In progress to 📋 Backlog in v2.0 Aug 6, 2026
alexander-yevsyukov and others added 7 commits October 1, 2026 18:05
…utt-1537c8

# Conflicts:
#	docs/_script/check-samples
Replace the v1.2.0 binaries in `docs/_bin` with the v1.2.4 release assets and
add the Windows binary for completeness. The release assets map to the
existing names as follows:

- `embed-code-linux` -> `embed-code-ubuntu`
- `embed-code-macos-arm64.zip` -> `embed-code-macos`
- `embed-code-macos-x64.zip` -> `embed-code-macos-x86_64`
- `embed-code-windows.exe` (new, name kept)

Each file matches the SHA-256 digest published with the release.

v1.2.0 logged argument errors, such as a `docs-path` that does not exist, but
exited with status 0, which let `checkSamples` pass while checking nothing.
Since v1.2.3 (upstream commit 9fa4244), every failure exits with status 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`embed-code` v1.2.4 exits with status 1 on argument errors, stale embeddings,
and missing code files, so the script can rely on the tool's exit status
again: `checkSamples` fails whenever the tool does, without parsing its output.

This restores the script to its `master` version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alexander-yevsyukov alexander-yevsyukov changed the title Make checkSamples fail loudly on embed-code errors Update embed-code to v1.2.4 so checkSamples no longer passes silently Oct 1, 2026
alexander-yevsyukov and others added 3 commits October 1, 2026 19:22
#71 moved the main (v1) documentation from `content/docs/1` to the root
`content/docs`, but `docs/_settings/v1.embed-code.yml` kept pointing at
`../content/docs/1`. Before `embed-code` v1.2.3 the resulting error was
swallowed and `checkSamples` passed while checking nothing; since then the
`checkSamples` and `embedCode` tasks fail on the missing folder.

Also drop the `embed-mappings` block: v1.2.3 renamed that key to `embeddings`,
so v1.2.4 ignores it and reads only the top-level paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the content of the main version moves, the code embedding config must
follow it. Moving the content without updating the config is what left
`docs-path` pointing at a missing folder after #71.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The page moved from `/docs/1/introduction/` to `/docs/introduction/` in #71.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@alexander-yevsyukov alexander-yevsyukov moved this from 📋 Backlog to 🏗 In progress in v2.0 Oct 1, 2026
@alexander-yevsyukov alexander-yevsyukov changed the title Update embed-code to v1.2.4 so checkSamples no longer passes silently Update embed-code to v1.2.4 so checkSamples no longer passes silently Oct 1, 2026
@alexander-yevsyukov
alexander-yevsyukov merged commit db6fabf into master Oct 2, 2026
5 checks passed
@alexander-yevsyukov
alexander-yevsyukov deleted the claude/nifty-northcutt-1537c8 branch October 2, 2026 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

2 participants